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This FOCUS Technical Brief summarizes a paper recently published in the open-access journal Implementation 
Science! (Stone & Lane, 2012). The full paper presents a conceptual framework that integrates all three 
knowledge-generating methods—scientific research (R), engineering development (D), and industry production 
(P)—into a logic model format, which is useful for planning, obtaining, and measuring the intended beneficial 
impacts by implementing knowledge in practice (Lane & Flagg, 2010). 

Background about development outputs (patented prototypes) 


and almost no discernible econometrics related to 
production outputs (commercial innovations). This 


Government-sponsored science, technology, and 
innovation (STI) programs support the socioeconomic 
aspects of public policies in addition to expanding the disparity is particularly problematic for the expressed 
knowledge base. For example, beneficial healthcare intent of such programs because the last category, 


services and devices are expected to result from production outputs, leads to most of the measurable 
socioeconomic benefits, such as improved health 


and function from product and service use, revenues 


investments in research and development (R&D) 


programs, an expectation that assumes a causal 
link to commercial innovation. Such programs generated by corporations, and tax payments into 


are increasingly held accountable for evidence of government coffers resulting from the new net wealth 


impact—that is, innovative goods and services generated in the commercial marketplace. 


resulting from R&D activity. 


Methods 
However, the absence of comprehensive models This issue of FOCUS summarizes a paper recently 
and metrics skews evidence gathering toward published in the open-access journal Implementation 
bibliometrics about research outputs (published Science! (Stone & Lane, 2012). The full paper presents 
discoveries), with less focus on transfer metrics a conceptual framework that integrates all three 


1. From: "Modeling Technology Innovation: How Science, Engineering, and Industry Methods Can Combine to Generate Beneficial Socioeconomic Impacts," by 
V. |. Stone and J. P. Lane, 2012, Implementation Science, 7(44). Copyright 2012 by authors. Adapted by SEDL in compliance with open-access permission 
under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and 
reproduction in any medium provided the original work is properly cited. Available from http://www.implementationscience.com/content/7/1/44 
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knowledge-generating methods—scientific research 
(R), engineering development (D), and industry 
production (P)—into a logic model format, which 

is useful for planning, obtaining, and measuring 

the intended beneficial impacts by implementing 
knowledge in practice (Lane & Flagg, 2010). 


The framework also integrates the Context- 
Input-Process-Product (CIPP) model of evaluation 
(Stufflebeam, 2004). The CIPP model takes a systemic 
approach by referring to project goals, inputs, 
processes, and outputs. Needs analysis, which is 
central to context evaluation, lends direction to 

a project by orienting it more toward the target 
audience’s needs while bringing relevance (worth) 
to the planned output. Input evaluation ensures that 
a project is feasible. Process evaluation for research, 
development, or production methods promotes 
efficiency and effectiveness. Output evaluation 
ensures and assesses the quality (merit) of the output 
and continues to follow up. Thus, the CIPP approach 
to evaluation builds relevance into STI policies and 
programs while sustaining appropriate focus on the 
issue of rigor (Worthen, Sanders, & Fitzpatrick, 1997). 


Results 


The resulting logic model framework explicitly 
traces the progress of knowledge from inputs as it 
moves through the three knowledge-generating 
processes (R, D, P) and their respective knowledge 
outputs (discovery, invention, innovation) to the 
intended sociobeneficial impacts. The framework 
is a hybrid model for creating technology-based 
innovations by merging best practices in new 
product development with a widely accepted 
knowledge translation (KT) approach. 


Given the emphasis on evidence-based practice 

in the medical and health fields and “bench to 
bedside” expectations for knowledge transfer, 
sponsors and grantees alike should find the model 
useful for planning, implementing, and evaluating 
innovation processes. 


Figure 1 on pages 4-5 presents the logic model 
framework, which shows the results of integrating 
the CIPP model with the three related methods of 


scientific research, engineering development, and 
industrial production. This hybrid model provides 
technology-based innovation program sponsors 
and project managers with an operational role for 
evaluation, from incorporating relevance along with 
rigor to tracking outcomes and impacts as well as 
activity outputs. 


Figure 1 combines all the relevant components in 
a comprehensive diagram to show how the role 

of evaluation spans the entire innovation process 
and how KT serves to bridge the components. 

The diagram illustrates the links between R, D, 

and P methods and how they combine to create 
and deliver a technology-based innovation to 

the marketplace. It also depicts the mechanisms 
involved in generating the socioeconomic benefits 
expressed in public policies and supported through 
government programs. 


The time and effort required to progress through 

this sequence is partly dependent on the path taken. 
One can use the figure to trace paths of differing 
length from the output of any method (R, D, or P) 

to the outcomes and impact. The time frame for 
research outputs to achieve impacts—particularly 
for technology-based projects—is longer because 

of the need to pass through the two downstream 
methods of development and production. For 
research projects, achieving impacts is often beyond 
the scope of funding and the project's time frame. 
This issue is an important point for project and 
program accountability, which typically tracks results 
only up to the termination of the funding time frame. 
It is unlikely that research projects can demonstrate 
downstream impacts during the award period. At 
best, researchers can demonstrate the downstream 
plan through which they or other stakeholders 

will complete the development and/or production 
activities and thereby transform outputs into impacts. 


The integrated logic model in Figure 1 emphasizes the 
importance of performing two context evaluations 
prior to initiating any efforts intended to generate 
technology-based innovations. This initial planning 
phase of a project is the opportune time to apply 

the two forms of context evaluation: (1) program 
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context—the analysis of the broader situational 
context around a project's identified problem, which 
informs funding priorities and requests for proposals; 
and (2) project context—the needs and opportunities 
analysis specific to a project's immediate context, 
which provides information necessary for defining 
project objectives. Using this approach helps ensure 
that evidence-based program priorities lead to the 
funding of evidence-based project objectives. 


The logic model highlights three additional elements: 


1. The CIPP evaluation activities are juxtaposed above, 
to the left, and below the project activities column 
and are connected respectively to its objectives, 
structure, and process. 


2. The project activities 
column shows R, D, and 
P activities and how they 
collectively advance 
new knowledge toward 
commercial innovations. 
Note the KT bridges 
within the project process 
and how they link 
methodological outputs 
vertically and horizontally. Note also the initial 
gate (G) before the R phase, where one can avoid 
the time and expense involved in sponsoring new 
research if the necessary knowledge already exists 
in the literature base inside or outside the field of 
application. Similarly, the transitions from research 
to development and then from development to 
production should be considered decision gates 
where one can opt to proceed or to stop work. 
This option prevents the mindset of proceeding 
regardless of the likely results simply because 
resources have been allocated for that purpose. 


3. The KT bridges go outward from outputs to 
outcomes; note the forked KT symbols going to 
short-term outcomes. Here, KT happens in two 
ways: a general KT (the white boxes in the Short- 
Term Outcomes column) and a more focused KT 
(the shaded boxes in the same column). The first 
case involves delivering outputs to all stakeholders 


The integrated logic model in Figure 1 
emphasizes the importance of performing 
two context evaluations prior to initiating 

any efforts intended to generate 


technology-based innovations. 


with potential interest. The second case 

involves perhaps limiting KT to a specific group, 
organization, or individual, such as a manufacturer, 
that is positioned to treat the knowledge as an 
input to the next method. 


Figure 1 also extends the logic model over time. 

For each KT case—general KT and focused KT— 

the diagram shows a sequence of outcomes that 
should result from the planned and coordinated 
outputs of R, D, and P activities. Programs and 
projects are expected to obtain a sequence of 
outcomes, from changing stakeholders’ awareness 
of specific knowledge, to building their interest in 
the knowledge, to their eventual implementation 

of the knowledge. Implementation should result in 
changes to practice (e.g., the use 
of evidence-based applications, 
prototype construction and testing, 
and commercial device and service 
manufacturing) or to policy (e.g., 
regulation and reimbursement of 
devices and services). 


The initial context evaluation 
mentioned earlier helps ensure 
that programs and projects achieve 
this sequence of outcomes leading to the expected 
impact. At the program level, evidence-based 
information about socioeconomic needs amenable 
to technology-based innovations helps funding 
agencies assess grant proposals for relevance, define 
indicators of impact, and determine how to monitor 
and evaluate funded projects. The context evaluation 
ensures that needs remain central to funding priorities 
and project deliverables. 


At the project level, the evidence-based needs 
analysis aligns the project deliverables with the 
sponsor's mission while ensuring the relevance of 
project outputs to the intended knowledge users 
prior to initiating activities. Figure 1 shows how 
this prior-to-grant perspective flows and suggests 
why this approach is preferable to end-of-grant 

or integrated approaches to KT, particularly for 
those programs and projects explicitly intended 
to result in beneficial socioeconomic impacts from 
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Figure 1. Planning and Evaluating Technology-Based R&D: 


Role of KT from Beginning to End 
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Note.The data in Figure 1 are from Figure 7 in "Modeling Technology Innovation: How Science, Engineering, and Industry Methods Can 
Combine to Generate Beneficial Socioeconomic Impacts," by V. |. Stone and J. P. Lane, 2012, Implementation Science, 7(44). Copyright 
2012 by authors. Adapted by SEDL in compliance with open-access permission under the terms of the Creative Commons Attribution 
License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium 
provided the original work is properly cited. Available from http://www.implementationscience.com/content/7/1/44 
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Figure 1 presents an overview of planning and evaluating a 
technology-based research and development (R&D) program. The 
overview explicitly summarizes the role of knowledge translation 
(KT) in increasing the likelihood of obtaining the intended beneficial 
impacts from project outputs. The model is structured around six 
columns sequentially connected by arrows suggesting progressive 
motion. Columns 1 and 2 refer to project activities and project 
output and show how KT is embedded in the interactions that result 
in outputs from research (R), development (D), and production (P) 
processes. Columns 3, 4, 5, and 6 present a detailed view of the KT 
connections through the progression from outputs (Column 2) to 
long-term impact (Column 6). This progression takes two alternate 
effect paths that cut across short-term (Column 3) and mid-term 
(columns 4 and 5) outcomes. The model shows the difference in time 
between the two paths for achieving an impact from an R output: 
the shorter path, where knowledge users (KUs) become aware of the 
output in the short term, and the longer path, where KUs proceed 
further by implementing knowledge to action (KtA) to achieve the 
intended technological innovation. Figure 1 captures the above 
concepts to show the role of KT in effective planning of technology- 


based R&D programs for impacts. 


Research 
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technology-based innovations. Given the option, 
why would program teams pursue any other path? 


Figure 1 also integrates key concepts and creates 
connections to guide the construction of integrated 
logic models for utilization-focused R&D. As a static 
graphic, the figure is of necessity simple and linear 
in form. Yet it can serve as a basis for constructing 
nonlinear and complex models, as needed, to 
incorporate and explicate elements that have a 
bearing on the causal sequence represented by this 
simple model. In addition, the figure can be readily 
expanded for individual programs and projects to 
reflect their unique characteristics and contexts. 


The main point of the 
integration and connection 
among concepts in Figure 1 

is to champion the cause 

of relevance, alongside 

rigor, through a continuous 
KT effort that starts at the 
beginning of an R&D program 
or project. Future additions 

to this work will address how to better integrate 
government and academic R&D programs with 


elements from the CIPP model of evaluation 


privately funded industry efforts to implement the 
outputs from R&D in technology-based innovations. 


The outcomes in the commercial marketplace 

are necessary as incentives for companies to 
generate the desired socioeconomic benefits. 
These incentives include revenues from sales paid 
to corporations to cover their costs (e.g., salaries, 
materials, and facilities) and profits to owners and 
shareholders. Of course, a portion of these revenues 
are paid to the government as taxes (profits to 
companies, income to employees, taxes on sales), 
which cycle back through the public coffers to be 
allocated as public funds used to sponsor R&D. 
Moreover, profitable companies benefit their home 
nations, so the balance of trade translates directly 
into national R&D capacity. 


The approach described here integrates 


into the basic, linear logic model format 
that currently guides program planning and 


evaluation practice. 


Summary 


Efforts to improve society while competing 
economically necessarily include programs that 
support technology-based innovations. Quality-of- 
life issues are paramount in the fields of medicine 
and healthcare. The process through which 
scientific knowledge is translated and technological 
knowledge is transferred should be accurately 
modeled for planning, implementation, and 
evaluation purposes. Describing the mechanisms 
underlying technology-based innovations and 
tracking the indicators of progress are necessary 
for establishing coherent milestones and 
accomplishing systematic results. If successful, 
sponsoring organizations will 
shift their perspective from 
the solution-driven “bench 

to bedside” approach to the 
need-driven “bedside to 
bench and back” approach. 


The approach described here 
integrates elements from the 
CIPP model of evaluation into 
the basic, linear logic model format that currently 
guides program planning and evaluation practice. 
What links the two models in this framework is the 
provision of a context evaluation prior to initiating 
any activity. This prior-to-grant perspective elevates 
the quality of relevance to a level equal to the 
quality of rigor—an orientation encompassing the 
stakeholders who determine success or failure of 
the entire effort. As a result, funding agencies can 
focus program goals to ground project objectives 
in the context of validated needs. The framework 
also clarifies the roles of process and product 
evaluations, which strengthen the merit and worth 
of project outputs. The role of outcome evaluations 
beyond the traditional measures of outputs (i.e., 
publications, patents) is to assess the actual 
socioeconomic impacts and deliver those evidence- 
based results to funders and stakeholders alike. 
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CENTER ON KT4TT 


The 5-year Center on Knowledge Translation for Technology Transfer (KT4TT) project (http://kt4tt. 
buffalo.edu) was awarded to the University at Buffalo (SUNY), Center for Assistive Technology (CAT) on 
October 1, 2008. SEDL and Western New York Independent Living, Inc., are partners in the project. SEDL's 
role focuses on utilization-oriented methods of dissemination, training, and technical assistance to 
effectively communicate with knowledge producers and knowledge users. This FOCUS Technical Brief is a 


product of the SEDL-KT4TT partnership. 


The project focuses on three key outcomes: 


« Improved understanding of the barriers preventing successful knowledge translation for technology 


transfer and ways to overcome these barriers 


« Advanced knowledge of best models, methods, and measures of knowledge translation and technology 


transfer for achieving outcomes 


- Increased utilization of these validated best practices by NIDRR’s technology-oriented grantees 
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Figure 1 presents an overview of planning and evaluating a 
technology-based research and development (R&D) program. The 
overview explicitly summarizes the role of knowledge translation 
(KT) in increasing the likelihood of obtaining the intended beneficial 
impacts from project outputs. The model is structured around six 
columns sequentially connected by arrows suggesting progressive 
motion. Columns 1 and 2 refer to project activities and project 
output and show how KT is embedded in the interactions that result 
in outputs from research (R), development (D), and production (P) 
processes. Columns 3, 4, 5, and 6 present a detailed view of the KT 
connections through the progression from outputs (Column 2) to 
long-term impact (Column 6). This progression takes two alternate 
effect paths that cut across short-term (Column 3) and mid-term 
(columns 4 and 5) outcomes. The model shows the difference in time 
between the two paths for achieving an impact from an R output: 
the shorter path, where knowledge users (KUs) become aware of the 
output in the short term, and the longer path, where KUs proceed 
further by implementing knowledge to action (KtA) to achieve the 
intended technological innovation. Figure 1 captures the above 
concepts to show the role of KT in effective planning of technology- 
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